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In today's competitive world, robot designs are developed to simplify and 
improve quality wherever necessary. The rise in technology and 
modernization has led people from the unskilled sector to shift to the skilled 
sector. The agricultural sector's solution for harvesting fruits and vegetables 
is manual labor and a few other agro bots that are expensive and have 


various limitations when it comes to harvesting. Although robots present 


may achieve harvesting, the affordability of such designs may not be 


Keywords: possible by small and medium-scale producers. The integrated robot system 
aa is designed to solve this problem, and when compared with the existing 
Affordability manual methods, this seems to be the most cost-effective, efficient, and 
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viable solution. The robot uses deep learning for image detection, and the 
object is acquired using robotic manipulators. The robot uses a Cartesian and 
articulated configuration to perform the picking action. In the end, the robot 
is operated where carrots and cantaloupes were harvested. The data of the 
harvested crops are used to arrive at the conclusion of the robot's accuracy. 
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1. INTRODUCTION 

The diverse climate and soil of India ensure the obtainability of all different types of fresh fruits and 
vegetables. India ranks in the production of fruits and vegetables second behind China. During 2015-2016 the 
National Horticulture database issued that India produced 169.1 million metric tonnes of vegetables and 90.2 
million metric tonnes of fruits. The fruits were cultivated at 6.3 million hectares, while the cultivation of 
fruits stood at 10.1 million hectares. When it comes to ginger and okra, India is the largest producer of 
vegetables and ranks second in the production of potatoes, brinjal, cabbage, cauliflowers, onions, and other 
vegetables. In fruits, India ranks first in the production of bananas (25.7%), mangoes (40.4%), and papayas 
(43.6%). Export has tremendous opportunities when it comes to the production of fruits and vegetables [1]. 
During 2019-20, India exported fruits and vegetables which is worth Rs.9,182.88 crores/1,277.38 USD 
Millions which consisted of and vegetables worth Rs.4,350.13 crores/608.48 USD Millions and fruits worth 
Rs.4,832.81 crores/668.75 USD Millions. Bangladesh, Oman, Qatar, Nepal, Malaysia, UAE, Netherland, the 
UK, and Sri Lanka, are the major areas where Indian fruits and vegetables are exported. The production rate 
of horticulture is increasing at an exponential rate. The input cost and land used are less in horticulture, and 
this factor is of great benefit. The fruits and vegetables provide a high nutritional value, and usage is higher 
among the urban population than rural population. The process involved in horticulture, such as harvesting, is 
a slow, unskilled, tedious, and repetitive job that can be automated. If this process is automated, farmers can 
work on improving their production yields which will further help the entire society. 
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2. RESEARCH METHOD 
2.1. Literature review 

Robotics in agriculture was first developed in the 1920s. The first step towards agricultural robotics 
was the mechanical harvesters. These use a simple and onerous mechanism to gather the harvest. The first 
mechanical harvest was used to gather tomatoes, which was patented in 1960. Further on, various harvesting 
systems came into play that has revolutionized this world. In terms of ground-level harvesting, the SW 6010 
is the first autonomous robot which is available that can collect strawberries. The other system is called soft 
harvesting, where delicate suction cups are used to collect the fruits such as pineapples, apples [2], [3]. These 
robotic arms can also be combined hand to hand to provide better accuracy while gathering. MetoMotion's 
robotic system is described as 'a multipurpose robotic intensive system for labor-intensive tasks in 
greenhouses which is to harvest tomatoes [4]. Vinken built is a robot that can see underground. It emits an 
electric signal which identifies the vegetables [5]. A tomato harvesting system in the form of a mobile robot 
is capable of harvesting with manual navigation [6]. Various systems exist that harvest strawberries on a 
semi-automation basis and chili pepper harvesting using the robotic arm [7]-[9]. All these systems are some 
of the ideas that are present in the market. 

The solutions in the market are extravagantly expensive and cannot be afforded by small-medium 
scale farmers. The labor assigned to take care of the farm must be skilled highly. The consumption of 
electricity increases as the robots run through electricity. The present harvesting robot requires manual 
control, but that is not a viable option as the task is to automate the harvesting task. If any technical error 
occurs, the entire harvesting comes to a halt until the system is fixed. This creates a delayed harvest. A 
completely automated farm is not a practical possibility as of now. 


2.2. Proposed solution 

The current solutions in the market either involve a mobile harvesting system or robotic arm 
harvesting, not a combination of both. No model has been executed with the concept of a Cartesian and an 
articulated robot. The design developed aims to make this system affordable at all levels of farming, and this 
can be operated at great ease [10]. No manual intervention is necessary for this system as this harvesting 
robot scans the entire area and yields the output. The accuracy of the yield is of the standard level. The 
products that can be harvested by this robot design includes cantaloupe, carrot, radish, saffron, pineapple, 
chili paprika, and all underground vegetable. The first step, as shown in Figure 1, involves the user starting 
the process through the display unit. As soon as the command is given, it sends the instruction to the cloud 
server, which communicates to the robot station to begin the process [11]. The robots then, from the docked 
station, travels to the initial point in the farm to start the harvesting. Once the harvesting is completed, the 
collected harvest is then delivered to the packing station by the robot. Then the robot travels to the charging 
point near the robot station and goes to the initial position, and awaits further instruction. A detailed outline 
of the system is shown in Figure 2. 
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Figure 1. Process of the system 


2.3. Vision system 

The vision system software for the detection of the fruits and vegetables for harvesting uses the 
YOLO-V3 (you only look once) object detection technique to identify and acquire the location of the fruits. 
YOLO-V3 uses a Darknet-53 feature extractor which has 53 convolutional neural network layers and skips 
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connections inspired by ResNet [12]. It also has a feature pyramid network that allows the YOLO-V3 to learn 
and detect objects of different sizes. The output of the feature pyramid network is known as a grid, and it has 
three scales of grids. In each of these grids, three anchor boxes with the same centroid of different sizes are 
placed on them. These anchor boxes predict the class id and the location of the object. Thus, the bounding boxes 
are applied to the image [13]. The YOLO-V3 object detection was chosen among many techniques from Figure 
3, which is plotted between common objects in context (COCO) average precision and inference time. It is clear 
that the YOLO-V3 has the average precision and speed when compared with other methods [14]. The 
main advantage of using a convolution neural network is that it can even detect a crop if it is covered with 
leaves [15], [16]. 

The YOLO-V3 network was trained using Google Colab [17], as it has a powerful graphics 
processing unit and more compute unified device architecture (CUDA) cores to reduce the overall training 
time [18]. It took around 5 to 6 hours for 2,000 iterations using 1,000 images of the required crop, which is to 
be detected 19], [20]. The iterations were performed for carrots and cantaloupe. The camera is 5-megapixel 
and is equipped with infrared lights. This is useful for the operation of the robot even during night time which 
leads to greater efficiency in the output. The graph plotted between loss and the no of iterations. After 2,000 
iterations, the loss came down almost near to zero. The basic principle is that more loss leads to less detection 
and less loss leads to an accurate detection. 
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Figure 2. Outline of the system Figure 3. Performance comparison graph 


2.4. Cartesian axis movement 

After locating the crop, the open source computer vision library (OpenCV) software sends the 
Cartesian coordinates to the universal G Code sender software. Now, this software converts these coordinates 
into G Codes. Then it is sent to the ATmega microcontroller through serial data transfer. The microcontroller 
acquires these G Codes and converts them to pulses for driving the stepper motor and to move the Cartesian 
robot to the specified location. After this process, the articulated robotic arm which is attached to the 
Cartesian robot harvests the crop. 


2.5. Navigation 

The localization of the robot is done through global positioning system (GPS) and navigation 
through waypoints. As it is an outdoor application, light detection and ranging (LIDAR) won't be suitable for 
this robot [21]. In this case, the QGroundControl software is for creating waypoints and navigating the 
automated guided vehicle. The concept of precision farming is used to improve accuracy [22], [23]. First, the 
boundaries of the farm are marked through the QGroundControl software, and then the waypoints are 
generated for path planning, as shown in Figure 4 [24], [25]. Once the path is planned, the entire area is 
covered by the robot to obtain all the grown harvest. The software ensures that only fully grown fruits and 
vegetables are harvested. 


2.6. Modules 

The design of the harvesting bot mainly involves four modules: 1) Automated guided vehicle 
(AGV), 2) Cartesian axis, 3) Articulated axis, and 4) Sensors and navigation. The computer-aided design of 
the harvesting robot is shown in Figure 5. 

The AGV consists of four 150 mm rubber gripped castor wheels for better friction and traction to 
the ground. The AGV linear movements are actuated by two 300 mm wheels with the 24-voltage brushless 
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direct current hub motor, and the shaft of the wheels is supported through ball bearings. Two lithium-ion 
batteries, which have a 24-voltage and current capacity of 10 Ah, are connected in parallel so that they can be 
used as the main power supply. The base frame is crafted through wood as the robot is a prototype and the 
wooden planks are easily affordable [26]. 


Figure 4. Path planning 


The Cartesian axis consists of 4 aluminium V-Slot channels of size 40 mm length, 20 mm width, 
and 1100 mm height are used as guideways for the V-Wheels [27]. The V-Wheels are attached to the 3D 
printed components, which act as an end support carrier for the V-Slot channels. The Cartesian axis also 
includes three lead screws for each X, Y, and Z-axis, which act as linear actuators and are coupled with the 
Nema 17 stepper motor using a flexible coupler. The flexible couple gives an advantage as a vibration 
damper. The ends of the lead screws are supported through 8mm bore diameter ball bearings. However, as 
the Z-axis carrier uses linear rods, neglecting the need for V-Slot channels so that linear bearings are used 
here to support the linear rods. The Articulated robot is a 3D printed six servo motors controlled robotic arm 
that uses Adafruit servo motor drivers for controlling the servo motors simultaneously. The filament used in 
3D printing is polylactic acid plastic (PLA) [28]. The robotic arm is designed through Solid works 3D modeling 
software, and Inverse kinematics is done through Denavit and Hartenberg parameters [29], [30]. The gripper 
designed is by considering the object size and by testing in various simulation platforms [31]-[33]. Motor sizing 
is the crucial part while designing the robotic arm as it determines the torque requirements, and it is done 
through oriental motor software. Ultrasonic sensors of 3 numbers are used here to avoid any obstacles while 
the automated guided vehicle is in motion. A stereoscopic image sensor is used here as an input to the image 
detection and classification software and also to find the depth of the image so that the robotic arm can reach 
the exact height of the crop and acquire it. For outdoor applications, as the robot is not bounded by any walls, 
the GPS waypoint navigation is used. The GPS waypoints navigations require inputs such as latitude, 
longitude, and the direction of the robot so that the global positioning system and compass modules are used 
here to localize the robot. All these modules are combined in order to give the end result of the final 
harvesting robot, as shown in Figure 6, which is the working model of the concept. 


Figure 5. Harvesting robot Figure 6. Final harvesting robot 
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3. RESULTS AND DISCUSSION 

The harvesting robot was tested in carrot and cantaloupe farming fields and gave us the following 
outputs, as shown in Table 1. The vision system was performed using the YOLOV3 algorithm. The robot was 
able to detect 232 out of 300 carrots with an average accuracy of 93% in 4 seconds. Similarly, 245 out of 300 
cantaloupes were detected, leading to an average accuracy of 95% in 2 seconds. It was also able to 
differentiate between the fully grown crops and the growing ones by feeding the neural network dataset with 
more images. 


Table 1. The output of accuracy and the time is taken for each crop 


No. Type of crop No. of crops Average accuracy Time taken to complete the detection 
detected (%) (seconds) 
1 Carrot 232 out of 300 93 4 
2 Cantaloupe 245 out of 300 95 2 


As the YOLOV3 algorithm also enables us to detect multiple classes, it can be used in multiple 
fields which contain different crops. The proposed robotic manipulator was able to acquire the crop in 
approximately 18 seconds. It is too high when compared to the time taken by a manual process, but still, as 
humans grow tiresome, this result can be acceptable due to the deviations in the robot navigation paths 
caused by the farming field's obstacles such as dirt and stones. The waypoint navigation by using a global 
position system gave an accuracy of around 87%. 


4. CONCLUSION 

An integrated robot system was designed, and the system was tested against several products using 
software in order to check its validity. A prototype of the designed system was implemented and verified in 
the form of hardware. In this development, automatic crop harvesting was performed through the method of 
position detection and harvesting using a robotic manipulator with a harvesting hand that does not damage 
the crop. The accuracy of the detection of crops such as carrots and cantaloupe was found, and the time taken 
for the detection of crops was verified. This robot may not replace a human but can work collaboratively as 
its performance doesn't exceed the levels of a human. This collaborative robot can be used in small-medium 
scale farms, which do not involve complete automation, yet the production rate increases. In the distant 
future, as unskilled labor gets eradicated, this harvesting bot will play a crucial role in fixing that problem. 
The current agricultural industry is going through hardship, and this product is set to make a breakthrough 
when it reaches the market as it is affordable and simple to use, which the farmers expect. 
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